System and method for 3D imaging using structured light illumination

ABSTRACT

A biometrics system captures and processes a handprint image using a structured light illumination to create a 2D representation equivalent of a rolled inked handprint. A processing unit calculates 3D coordinates of the hand from the plurality of images and maps the 3D coordinates to a 2D flat surface to create a 2D representation equivalent of a rolled inked handprint.

CROSS REFERENCE TO RELATED PATENTS/PATENT APPLICATIONS ContinuationPriority Claim, 35 U.S.C. §120

The present U.S. Utility Patent Application claims priority pursuant to35 U.S.C. §120, as a continuation, to the following U.S. Utility PatentApplication which is hereby incorporated herein by reference in itsentirety and made part of the present U.S. Utility Patent Applicationfor all purposes:

1. U.S. Utility patent application Ser. No. 13/527,704, entitled “Systemand Method for 3D Imaging using Structured Light Illumination,” filedJun. 20, 2012, pending, which claims priority pursuant to 35 U.S.C.§120, as a divisional application, to the following U.S. Utility PatentApplication which is hereby incorporated herein by reference in itsentirety and made part of the present U.S. Utility Patent Applicationfor all purposes:

2. U.S. Utility patent application Ser. No. 11/586,473, entitled “Systemand Method for 3D Imaging using Structured Light Illumination,” filedOct. 25, 2006, now U.S. Pat. No. 8,224,064 issued on Jul. 17, 2012,which claims priority pursuant to 35 U.S.C. §119(e) to the followingU.S. Provisional Patent Application which is hereby incorporated hereinby reference in its entirety and made part of the present U.S. UtilityPatent Application for all purposes:

-   -   a. U.S. Provisional Patent Application Ser. No. 60/730,185,        filed Oct. 25, 2005, now expired.

U.S. Utility patent application Ser. No. 11/586,473 also claims prioritypursuant to 35 U.S.C. §120, as a continuation-in-part (CIP), to thefollowing U.S. Utility Patent Application which is hereby incorporatedherein by reference in its entirety and made part of the present U.S.Utility Patent Application for all purposes:

1. U.S. Utility patent application Ser. No. 10/444,033, entitled “Systemand Technique for Retrieving Depth Information about a Surface byProjecting a Composite Image of Modulated Light Patterns,” filed May 21,2003, now U.S. Pat. No. 7,440,590 issued on Oct. 21, 2008.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

The U.S. Government has a paid up license in this invention and theright in limited circumstances to require the patent owner to licenseothers on reasonable terms as provided for by the terms of Contract No.2004-IJ-CX-K055 awarded by the National Institute of Justice, throughsubcontract with Eastern Kentucky University Contract: 06-202.

BACKGROUND OF THE INVENTION

1. Technical Field of the Invention

This invention relates to biometrics, and in particular to threedimensional (3D) imaging using structured light illumination forbiometrics.

2. Description of Related Art

Biometrics is the science of measuring and analyzing biological data. Inlaw enforcement and security fields, biometrics is used to measure andanalyze human features, such as fingerprints, facial patterns, handmeasurements, retinas, etc. Well known biometric measurements arefingerprints and palm prints. Fingerprints and palm prints are now and,for the foreseeable future the most relied upon biometric measurementsfor verifying a person's identity and also for linking persons to acriminal history and background checks. Criminal justice agencies relyon fingerprints and palm prints for positive identification to latentprints collected as evidence at crime scenes and in processing personsthrough the criminal justice system.

The National Institute of Science and Technology (NIST) and the AmericanNational Standards Institute (ANSI) supports the ANSI/NIST-ITL 1-2000Data Format for the Interchange of Fingerprint, Facial, & Scar Mark &Tattoo (SMT) Information. This standard defines the content, format, andunits of measurement for the exchange of fingerprint, palm print,facial/mug shot, and scar, mark, & tattoo (SMT) image information thatmay be used in the identification process of a subject. The informationconsists of a variety of mandatory and optional items, includingscanning parameters, related descriptive and record data, digitizedfingerprint information, and compressed or uncompressed images. Thisinformation is intended for interchange among criminal justiceadministrations or organizations that rely on automated fingerprint andpalm print identification systems or use facial/mug shot or SMT data foridentification purposes. Other organizations have different standards aswell for the content, format or units of measurement for biometricinformation.

The traditional method of finger print acquisition to meet suchstandards is to roll an inked finger onto a paper sheet. This method ofrolling an inked finger onto a paper sheet converts the inked 3D fingerprint into a two dimensional (2D) image on the paper sheet. The 2D imageof the inked 3D finger print is then converted into an electronicversion, such as by scanning. The electronic fingerprint and palm-printimages meeting specified standards allow for the rapid search ofmatching print images in extremely large databases of existingfingerprint and palm-print based records. For example, the FBI maintainsan Interstate Identification Index System for fingerprints and palmprints.

Though the need for accurate and fast biometric identification isincreasing, the above described known process of rolling an inkedfingerprint has many limitations. The rolled ink print technique is slowand cumbersome and often produces finger prints and palm prints of poorquality. It requires a trained technician to grasp and manipulate aperson's finger or hand, and even then it may take multiple attempts tosuccessfully capture a print that meets industry standards. The rolledfinger prints and palm prints can only be captured one at a time thuscreating a very slow image capture process that may take 5 to 10 minutesor more. Small amounts of contamination or excessively dry or moist skincan hamper or even preclude the capture of an acceptable image. Fingerprints and palm prints of some persons with fine or worn friction ridgescannot be captured. These disadvantages create a high acquisition andmaintenance cost that has significantly limited the widespread use ofbiometric identification based on finger prints and palm prints.

Thus, a need has arisen for a more robust, fast and accurate system forbiometric identification. In specific, a need has arisen for a systemfor hand print or finger print identification using biometrics that isfast, easy to use and accurate. In addition, a need has arisen for suchsystem to be able to capture and process such images to meet current andfuture industry standards.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

For a more complete understanding of the present invention, and theadvantages thereof, reference is now made to the following descriptionstaken in conjunction with the accompanying drawings, in which:

FIG. 1 illustrates an embodiment of a structured light illuminationsystem of the present invention.

FIGS. 2 a and 2 b illustrate an embodiment of the biometrics system ofthe present invention.

FIG. 3 illustrates another embodiment of the biometrics system of thepresent invention.

FIG. 4 illustrates another embodiment of the biometrics system of thepresent invention.

FIGS. 5 a and 5 b illustrate a more detailed view within a scan volumein one embodiment of the biometrics system of the present invention.

FIG. 6 illustrates one embodiment of a method for the biometrics systemto capture handprint images of the present invention.

FIGS. 7 a and 7 b illustrate one embodiment of a configuration of thecameras and projection unit in the biometrics system of the presentinvention.

FIG. 8 illustrates another embodiment of the biometrics system withmultiple projection units of the present invention.

FIGS. 9 a and 9 b illustrate one embodiment of a backdrop pattern andfiducials used for calibration and alignment in the biometrics system ofthe present invention.

FIGS. 10 a and 10 b illustrate embodiments of the image capture processof the biometrics system of the present invention.

FIGS. 11 a, 11 b, 11 c and 11 d illustrate embodiments of structuredlight patterns that may be used in the image capture process ofbiometrics system of the present invention.

FIG. 12 illustrates one embodiment of a method for image processing inthe biometrics system of the present invention.

FIG. 13 illustrates an image of a handprint with partitions forprocessing of the present invention.

FIGS. 14 a, 14 b, 14 c and 14 d illustrate one embodiment of a methodfor image processing in the biometrics system of the present invention.

FIG. 15 illustrates an effective two dimensional resolution of an imageas a function of angle of a surface of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The present invention is best understood in relation to FIGS. 1 through15 of the drawings, like numerals being used for similar elements of thevarious drawings. The following description includes various specificembodiments of the invention but a person of skill in the art willappreciate that the present invention may be practiced withoutlimitation to the specific details of the embodiments described herein.

One approach to creating a 3D image is called a structured lightillumination (SLI) technique. In SLI technique, a light pattern isprojected onto a 3D object surface. FIG. 1 shows an example SLI system10. In FIG. 1, the SLI system 10 includes a camera 18 and projector 20.The 3D object 14 is placed at a reference plane 22 that is apredetermined distance L from the projector 20 and camera 18. In thisexample, the projector 20 and camera 18 are in the same plane withrespect to each other to simplify calculations, but such positioning isnot required.

In use, the projector 12 projects a structured light pattern onto the 3Dobject surface 16. The structured light pattern can be a series ofstriped lines or a grid or other patterns, as discussed below. When thestructured light pattern is projected onto the 3D object surface 16, itis distorted by the 3D object surface 14. The camera 18 captures one ormore images of the 3D object surface 16 with the distortions in thestructured light pattern. The one or more images are then stored in animage file for processing by the image processing device 12. In someembodiments of the present invention, multiple structured light patternsare projected onto the 3D object surface 16 by the projector 20, andmultiple images of the 3D object with the structured light patterns arecaptured by the camera 18 or by other cameras added to the system shownin FIG. 1.

During processing of the image files, the distortions in the structuredlight pattern are analyzed and calculations performed to determine aspatial measurement of various points on the 3D object surface withrespect to the reference plane 22. This processing of the images useswell-known techniques in the industry, such as standard range-finding ortriangulation methods. The triangulation angle between the camera andprojected pattern causes a distortion directly related to the depth ofthe surface. Once these range finding techniques are used to determinethe position of a plurality of points on the 3D object surface, then a3D data representation of the 3D object 16 can be created. An example ofsuch calculations is described in U.S. patent application Ser. No.10/444,033, entitled, “System and Technique for Retrieving DepthInformation about a Surface by Projecting a Composite Image of ModulatedLight Patterns,” by Laurence G. Hassebrook, Daniel L. Lau, and Chun Guanfiled on May 21, 2003, which is incorporated by reference here.

Much research has been conducted on the type of structured lightpatterns to use in SLI techniques. For example, at first a single stripescanning system was proposed by J. A. Beraldin, M. Rioux, F. Blais, G.Godin, R. Baribearu, in “Model-based calibration of a range camera,proceedings of the 11^(th) International Conference on PatternRecognition: 163-167, the Hagure, the Netherlands (1992). Then, multiplestripe patterns with stripe indexing were proposed, see for example, C.Rochini, P. Cignoni, C. Montani, P. Pingi and R. Scopigno, “A low cost3D Scanner based on Structured Light, Computer Graphics Forum”(Eurographics 2001 Conference Proc.), vol. 20 (3), 2001 pp. 299-308,Manchester, 4-7 Sep. 2001.

In “Channel Capacity Model of Binary encoded structured light-stripeillumination,” by Raymond C. Daley and Laurence G. Hassebrook, inApplied Optics, Vo. 37, No. 17, June 1998, which is incorporated byreference herein, a technique is presented to enhance lateral resolutionby multiplexing the light structure to produce interlaced and encodedimages. In, “Composite Structured Light Pattern for Three-DimensionalVideo,” by C. Guan, L. G. Hassebrook and D. L. Lau, Optics Express, Vol.11, No. 5 dated Mar. 10, 2003, which is incorporated by referenceherein, a composite image comprising a plurality of modulated structuredlight patterns is described. Such composite image allows for recovery ofthe 3D depth map with a single image.

This analysis of relating distortion to surface points and use ofvarious structured light patterns is a crucial part of the processing ofthe 2D captured images to create a 3D model. The present inventionprovides an optimized system and method for capture and processing ofhandprints and other biometric features using SLI techniques in an easyto use and cost effective manner.

FIGS. 2 a and 2 b illustrate one embodiment of the present invention.FIG. 2 a illustrates a side view of a biometrics system 100 while FIG. 2b illustrates a front perspective view of the biometrics system 100. Thebiometrics system 100 uses SLI techniques to obtain images of a handprint as described herein, wherein a hand print includes either anentire hand print or a finger print or a thumb print or a palm print ora combination thereof.

The biometric system 100 includes a hand port 102 for insertion of aleft hand or right hand. The hand port 102 includes a scan volume 104.In the embodiment of FIG. 1, the hand is positioned in the scan volume104 with the palm down and with the back of the hand positioned on or infront of the backdrop 106. The backdrop 106 acts as the reference plane22 shown in FIG. 1. The opening of the scan volume 104 can be a cloth ordark plastic that allows entry of a hand but helps prevent ambient lightfrom entering the scan volume 104. The scan side 108 of the scan volume104 is a transparent material, such as plastic or glass. Alternatively,the scan side 108 may be left open with no material between the hand andthe SLI equipment below.

The SLI equipment includes one or more cameras 110 for capturing one ormore hand print images of all or portions of a hand positioned withinthe scan volume 104. The cameras are preferably commercial highresolution digital cameras or may be specialty cameras designed andmanufactured for the biometric system 100.

In addition, a projection unit 112 is positioned to illuminate all or aportion of a hand positioned within the scan volume 104 with one or morestructured light patterns. Additional projection units 112 may also beincorporated into the biometric system 100 as explained in more detailbelow. A display 114, such as an LCD screen or other display type,provides a display of the scan volume 104. The display 114 is preferablypositioned so that a subject can comfortably insert their hand and viewthe display 114 at the same time. The biometric system 100 is controlledby processor 116. Processor 116 may be a small personal computer orspecialty processor unit that is connected to the cameras 110 andprojection unit 112 through connections to USB ports on the cameras andprojector, Ethernet LAN connections or other type of connection.

As seen in FIG. 2 b, the display unit 114 displays the scan volume 104and in this example, a right hand is positioned within the scan volume104. One or more hand positioning pegs 118 are attached to the backdrop106. The pegs 118 assist in proper placement of a hand within the scanvolume 104 such that the field of views of the cameras 110 andprojection unit 112 cover at least a portion of the hand. A first peg118 a is positioned to guide placement of a thumb on a right hand. Asecond peg 118 b is positioned to rest between the second and thirdfingers of either a left hand or right hand. The third peg 118 c ispositioned to guide placement of a thumb on a left hand. A person ofskill in the art would appreciate that one or more additional pegs orother hand positioning guides may be used to help guide properpositioning of a hand within the scan volume 104.

Each of the above parts is illustrated in FIGS. 2 a and 2 b in anenclosure 120. This arrangement within the enclosure 120 provides for acompact system that limits ambient light within the scan volume 104. Theenclosure 120 also includes one or more adjustable supports 122 whoseheight may be adjusted to change the height of the enclosure 120 and somove the scan volume 104 up or down. This adjustment in height allowsthe biometrics system 100 to be in a comfortable position for a subjectto insert their hand into the hand port 102. For example, if theenclosure 120 is placed on the ground, it may be more convenient toinsert your hand into the hand port 102 if the height is increased. Ifthe enclosure 120 is placed on a table top, it would be more comfortableto decrease the height of the hand port 102. Thus, the height of theenclosure 120 can be adjusted using the support legs 122. A person ofskill in the art would appreciate that other height adjustmentmechanisms may be used as well.

Though shown in an enclosure 120, a person of skill in the art wouldappreciate that one or more of the parts of the biometrics system 100may be physically separated. For example, the display may be a separateLCD display connected by a cable to the cameras 112 and processor 116.The processor 116 may also be a separate PC or other processing devicenot incorporated within the enclosure 120, for example such that anoperator may control the biometric system 100 with an externalprocessor. In addition, the cameras 110 and projection unit 112 may beseparate physical units positioned around a scan volume 104.

FIG. 3 illustrates another embodiment of the biometrics system 100. Inthis embodiment, the biometrics system 100 is configured to allow a handto be positioned with palm facing upwards. Thus, the backdrop 106 willform the bottom or lower side of the scan volume 104 and the scan side108 will be the upper side. The projection unit 112 and cameras 110 willbe positioned above the scan side 108. The embodiment of FIG. 2 with ahand position of a palm down may be preferred because persons withmobility problems may not be able to rotate their hand for a palm upposition. A person of skill in the art would appreciate that otherconfigurations can include a swivel stand for the scanner that wouldallow adjustable rotation of the scan volume to 90 degrees or a full 180degrees rotation. Such rotation would allow for adjustment so that asubject with limited mobility of their hand may comfortably use thebiometrics system in any position or angle.

FIG. 4 illustrates another embodiment wherein the biometric system 100includes two tandem scanners, one for the left hand and one for theright hand. A processor 116 may operate both systems. Two displays 114 aand 114 b may be used to display the left and right hand respectively ora single display may be used to display both hands. The biometricssystem 100 in FIG. 4 allows for quick capture of handprints of both theleft and right hand of a subject concurrently.

FIGS. 5 a and 5 b provide a more detailed view within the scan volume104. One of the problems with SLI techniques is that the surfaces of the3D object must be within the fields of view of the one or more camerasand projectors. Since a hand is 3D object, it has many curves,discontinuities and angles. Thus, it is difficult to place all surfacesof the hand within the fields of view of the cameras 110 and projectionunit 112 when the hand is in one position. However, having to move thehand to multiple positions and capturing images in multiple positionswould slow down the capture process.

One embodiment of the present invention solves this problem by utilizingmirrors that are within the fields of view of the cameras 110 andprojectors 112 to reflect one or more hidden surfaces of the hand. Forexample, the scan volume 104 in FIGS. 5 a and 5 b illustrates a hand 124that is positioned against the backdrop 106. Assuming the fields of viewof the cameras are at the same angle shown, though the fingertip of theindex finger of the hand 124 is visible, the complete thumb tip is notvisible. So a mirror 130 is positioned on the backdrop 106. The mirror130 is reflecting the thumb tip so it is visible in the reflection ofthe mirror. A thumb rest 132 helps to position the thumb for reflectionin the mirror 130. The mirror 130 may also be recessed to reducecontamination from contact with a subject's thumb. Since the reflectionof the thumb in the mirror is within the field of view of the one ormore cameras 110 and projectors 112, wrap around scanning can occur ofthe thumb without having to adjust the position of the thumb.

In one embodiment of the biometrics system 100, the mirrors 130 havefilters to only reflect one or more certain colors, e.g. light ofcertain wavelengths such as red, green, or blue. Since the projectionunit 112 is projecting a structured light pattern onto the hand of thesubject, the reflection of the mirror may interfere with the projectionfrom projection unit 112, or it may be difficult to determine thepattern from the projection unit 112 versus the pattern reflected fromthe mirror 130. By placing a filter onto the mirrors 130, the mirrors130 can be designed to reflect only a certain color such as red, greenor blue. If multiple mirrors are implemented, each mirror may bedesigned to reflect a different color. Thus, the patterns reflected bythe mirror can be discerned separately from the projections of theprojection unit 112.

Alternatively or in conjunction with use of mirrors, a first set of oneor more cameras 110 and one or more projectors 112 may be positioned atan angle ideal to capture the thumb print while a second set of one ormore cameras and one or more projectors are positioned at an angle tocapture the fingerprints. Though this solution may be more difficult andrequire more complex processing to stitch together a complete 3D modelof an entire hand, it may have advantages when only separate images ofthumbprints and fingerprints are needed.

The biometric system 100 can be operated in either an autonomous entryor operator controlled entry mode. In autonomous entry mode, thebiometric system 100 does not require an external operator or control.The biometric system 100 can include an initiate button for a subject toinitiate a hand scan or the biometric system can operate in a previewmode that continuously monitors for a hand image. In an operatorcontrolled entry mode, an operator assists in operation of the biometricsystem 100 by assisting in correctly positioning the hand of a subjectwithin the scan volume 104, initiating a scan, or verifyingidentification from a subject. The operator may also assist inprocessing the scan images and verifying valid images were obtained fromthe subject.

FIG. 6 illustrates one embodiment of a method for the biometrics system100 to capture handprint images. In an autonomous entry mode or operatorcontrolled entry mode, the biometrics system 100 monitors for insertionof hand or initiation by an operator or subject controlled input orother type of input. Once a subject inserts a hand into the scan volume104 as shown in step 202, the biometrics system 100 begins a previewmode, as shown in step 204. During preview mode, the biometrics system100 assists in and verifies correct hand positioning. Using the handpositioning pegs 118, the subject tries to correctly position theirhand. The biometric system 100 captures low resolution images of thescan volume 104 as shown in step 206 and displays the images on thedisplay 114. The subject can view their hand and the pegs 118 in thedisplay 114 to assist in their hand positioning. The low resolutionimages are also acquired and processed by the biometric system 100 todetermine correct hand positioning, as shown in step 210. For example inthis step, the biometrics system 100 determines whether the placement ofthe tips of the fingers and thumb are within correct fields of view ofthe cameras 110 and projection unit 112.

As shown in step 212, the biometrics system 100 will prompt the subjectwith instructions if the hand is not correctly positioned with possibleocclusion of part of the hand from the cameras 110 or projection unit112. Such prompts may be verbal through an automated interactive voiceresponse unit and microphone in the biometrics system 100. Or theinstructions may be displayed on the display 114. Alternatively, theinstructions may be provided to an operator who assists the subject inhand positioning. The method for providing such instructions andspecific prompting instructions may be programmed or customizable by anoperator as well.

The biometrics system 100 will continue to operate in preview modecapturing low resolution images and providing instructions until itdetermines that the subject's hand is correctly positioned within thescan volume 104. When the hand is correctly positioned, the biometricssystem 104 then provides instructions to maintain hand position, asshown in step 214. The biometrics system 100 captures the hand printimages needed for processing as shown in step 216. This step 216 isexplained in more detail below with respect to FIG. 9. The biometricssystem 100 then processes the images to determine if viable images werecaptured. If for some reason viable images were not captured, e.g. thehand was moved or an error occurred in surface reconstruction orotherwise, the biometrics system 100 detects such errors duringprocessing of the images in step 218. It will then provide instructionsfor a second scan, as shown in step 220. The process will then return topreview mode again to assist in hand positioning for the second scan.Such process will continue until viable images have been captured. Theviable images are then processed in step 222. The processor 116 mayprovide such processing as needed at the time to determine viable imagesand complete processing later. Or if identification is neededimmediately for security reasons or to provide entry or access, thenprocessing is completed then. If processing may be performed at a latertime, images may be stored and processed by processor 116 or by anothercentral computer as requested by an operator.

Though the process has been described with respect to handprint images,a person of skill in the art would appreciate that the biometrics system100 and method described above could be used to capture images of otherfeatures.

FIGS. 7 a and 7 b illustrate one embodiment of the configuration of thecameras 110 and projection unit 112 in the biometrics system 100. InFIG. 7 a, the scan volume 104 is illustrated with a hand 124 positionedwithin the scan volume 104 on the backdrop 106. The arrangement of thecameras 110 and projection unit 112 below the scan side 108 is shown indiagram form. In this embodiment of FIG. 7 a, five cameras 110 a, 110 b,110 c, 110 d and 110 e are arranged around a projection unit 112. Thoughthe cameras 110 a-e and projection unit 112 are shown in a certainarrangement in FIG. 7 a, a person of skill in the art would appreciatethat other arrangements may be used as well.

The fields of view of the five cameras in FIG. 7 a are illustrated inFIG. 7 b. The cameras 110 a-e have been positioned such that theirfields of view (FOV) cover the surface of the hand positioned within thescan volume 104. FOV A-Left 300 and FOV A-Right 302 capture the fingerimages. Either FOV B-left 304 or FOV B-right 306 capture the thumb andmirror images of the thumb, depending on whether the right or left handis positioned in the scan volume 104. FOV C 308 and overlapping portionsof FOV B-left 304 and FOV B-right 306 are used to obtain the palmimages. As seen in FIG. 7 b, some of the fields of view overlap. Whenfields of view overlap, the resolution and signal to noise ratio (SNR)may be improved in the processing of the images.

Several parameters are used to determine the number of cameras 110, andas such the number of fields of view, and the size of the fields ofview. For example, the resolution in pixels per inch (ppi) andmodulation transfer function (MTF) of the resulting images are importantfactors. To meet certain industry or application standards, the camerasmust meet certain resolution, depth of field and MTF requirements. Inthis embodiment, each field of view is approximately a 5″ by 4″ regionwith the cameras having 2592 by 1944 pixels per inch (PPI) in order toobtain handprint images of at least 500 ppi. A person of skill in theart would appreciate that the resolution of the camera, MTF and size ofthe field of view affect the resolution of the resulting handprintimages. So in order to obtain a specified resolution, these parametersmust be designed accordingly. For example, fewer cameras with higherresolution and even larger fields of view may be used to obtain similarresolutions of the hand print images. Alternatively, more cameras at thesame or less resolution with smaller fields of view may also be used toobtain similar resolutions of the hand print images. Another factor toconsider is the cost of the system. More low cost, commerciallyavailable cameras with lower resolution may be more cost effective inthe design than fewer, specialty high resolution cameras.

Another consideration in type of camera is the angles or curvatures ofthe surfaces. As seen in FIG. 15, the effective two dimensionalresolution of an image is a function of angle. A camera 802 captures animage of a 3D object 804. When the 3D surface is relatively flat assurface 808, then the pixels per inch of the camera resolution is thesame as the pixels per inch of surface captured in the image. However,for a curved or angled surface 806, the pixels have more distancebetween them across the surface. The 2D resolution drops in relation tothe cosine of the angle of the surface. The following equation providesthe effective resolution for surfaces:

0.25*PPI Resolution of Camera*Cos(max_angle of surface)

Thus, the resolution of the cameras 110 need to be selected in view ofthe angles of the surfaces for a handprint or other 3D object used inthe biometrics system 100.

The embodiment of the biometrics system 100 shown in FIG. 7 a includesone projection unit 112 positioned between the cameras 110 a-e. Theprojection unit 112 projects the structured light pattern onto the handduring capture of images in order to construct a 3D model using SLItechniques, as discussed above. The projection unit 112 may be laserprojector, CRT projector or digital projector. These types of projectorshave an advantage that one such projector may project several differentstructured light patterns onto a surface as needed. Alternatively, theprojection unit 112 may be a consumer or industrial flash with aspecialized projection lens. The projection lens includes a structuredlight pattern slide for projecting onto a surface, as described in U.S.Provisional Application 60/744,259 filed on Apr. 4, 2006, “3 DimensionalImage Capture,” with inventor Paul Herber, which is incorporated byreference here. Such a flash projector may be useful due to its speed,brightness and compactness.

FIG. 8 illustrates another embodiment of the biometrics system 100 withmultiple projection units 112 a, 112 b and 112 c. Use of the multipleprojectors 112 a, 112 b and 112 c has an advantage that each projectionunit 112 in FIG. 8 may project the structured light pattern at differentangles within the scan volume 104 to cover more of the hand surface. Inorder to reconstruct the 3D surface representation using SLI techniques,the structured light pattern must be projected onto the surface andcaptured in an image. With one projection unit 112, some surfaces maynot be within its field of view, such as the sides of the fingers andcurves around the thumb. The multiple projection units 112 are able toproject and cover more surface areas of the hand within the scan volume104. Though the cameras 110 a-e are in slightly different positions thanin FIG. 7 a, they may be angled and focused to have similar fields ofview as shown in FIG. 7 b. Of course, a person of skill in the art wouldappreciate that different positions of fields of view may be designeddepending on the cameras and coverage desired.

FIGS. 7 and 8 also illustrate a background pattern 310 on the backdrop106. The background pattern 310 has a vital role in calibration andprocessing of the hand print images in biometrics system 100 and isshown in more detail with respect to FIGS. 9 a and 9 b.

The background pattern 310 preferably includes one or more types ofpatterns as shown in FIG. 9 a. First, the background pattern includes a“non-repeating” or fine pattern 312 such that a portion of the pattern312 is distinct in any particular area. The fine pattern 312 is used forfine calibration and alignments. The fine pattern 312 is preferably apseudo-random digital noise pattern or may be other types of patternsthat provide distinct features. Some repetition may be included in thefine pattern 312 as long as the pattern is sufficiently distinct toprovide alignments of different fields of view or partitions andcalibration. Thus, the fine pattern means any type of pattern, regularor noise pattern that allows any point within the pattern to be uniquelydetermined by edge characteristics, labeling, fiducial proximity, orother characteristics. Second, the background pattern includes one ormore fiducials 314 for course alignments and calibration. Third, thebackground pattern 310 includes projection areas 316 for patternprojection calibrations. The projection areas 316 are preferably whiteor a solid color and used to determine the intensity of color of thestructured light patterns on the background. Thus, the effect of thecolor of skin of the hand surface on the intensity of the structuredlight pattern can be determined. The background pattern 310 is threedimensional with well defined tier and surface heights. This isnecessary to calibrate the scan volume accurately. The depth range ofthe background pattern 310 should span that of the scan volume depth orthe depth of a portion of the 3D object to be scanned for most optimumcalibrations. The background pattern 310 is in black and white or otherhigh contrast colors to differentiate the pattern. During provisioningof the biometrics system 100, the exact world coordinates of thebackground pattern in the point cloud of the scan volume 104 aremeasured as well as intensity of the structured light pattern incaptured images. These reference measurements are thus known andpredetermined before a hand scan.

When the hand print images are captured, the background pattern 310 ispart of the field of view of the cameras 110 and so incorporated intothe handprint images. The known world coordinates and intensity of thebackground pattern 310 from the reference measurements are comparedduring processing of the handprint images into a reconstructed 3Dsurface. Calibration parameters are then modified to match the knownreference measurements during the surface reconstruction. For example,the processing unit may make adjustments to the calibration parametersfrom calculating coordinates of the backdrop pattern 310 in a handprintimage and comparing them with the predetermined coordinates of thebackdrop pattern. Thus, calibration can be performed as part of theprocessing of each and every hand scan.

In addition, the intensity of color of the structured light patterns onthe projection areas 316 can be determined for each hand scan.Differences in ambient light or projection units' intensity over timemay alter the intensity of color of the structured light patterns. Byknowing the pattern projection on the white projection areas, the coloror shading of the skin of the hand surface can be determined. The albedoor effect of the color of skin of the hand surface on the intensity ofthe structured light pattern can be compensated for during processing.

Using the background pattern 310 for calibration is superior to use offiducials for calibration alone. FIG. 9 b illustrates a fiducialstructure 318 that may be used for calibration. In the fiducialstechnique, fiducials or markers at known distances are imaged atprovisioning to determine calibration parameters. However, thisfiducials technique is too cumbersome and time consuming to be performedbefore each and every hand scan. So it can not compensate for drift inthe equipment setup and positions or affects from auto focusing or otherdistortions that may occur over time.

In addition, the use of the known world coordinates of the backgroundpattern 310 can be used to align or stitch together the various imagesfrom the cameras 110. Since the background pattern 310 is a random noisepattern, the unique position of a portion of the backdrop pattern 310 inan image can be matched to its overall position in the backgroundpattern. Thus, these known world coordinates of the background pattern310 in each image can be used to stitch together or align the imageswith respect to the background pattern 310. Any ambiguities may beresolved by matching details in the handprints such as matching ridgelocations. Thus, the background pattern 310 has advantages in both thecalibration and alignment processing of the handprint images.

The operation of one embodiment of the biometrics system 100 to capturehandprint images is now explained in more detail with respect to FIG. 10a. As shown in FIG. 6 step 216, the one or more cameras 110 andprojection units 112 capture the handprint images. FIG. 10 a illustratesone embodiment of this image capture process of step 216 in more detail.Prior to this image capture process, it is assumed that the hand hasbeen correctly positioned within the scan volume 104.

In the first step of the image capture process of the embodiment in FIG.10 a, the first projection unit 112 a projects a first structured lightpattern within the scan volume 112 and onto the surface of the handpositioned therein. Each of the one or more cameras 110 captures animage of the hand from its respective field of view. The cameras 110 maycapture such images concurrently to save time and before movement of thehand. The projection unit 112 must be calibrated to project thestructured light pattern for a period at least equaling the acquisitionwindow of all the cameras 110. If only a first structured light patternis being used for the SLI technique, then the next projector projectsthe structured light pattern. Or if only one projector is in use, asshown in the embodiment of FIG. 7, then the image capture process ends.

If more than one structured light pattern is being used for the SLItechnique, than the first projection unit 112 a projects a secondstructured light pattern within the scan volume 104 and onto the surfaceof the hand positioned in the scan volume. The one or more cameras 110again each capture an image of the hand from their respective field ofview while the second structured light pattern is projected thereon.This process continues until the first projection unit 112 a hasprojected each structured light pattern needed for the SLI technique andthe cameras have captured an image of the hand with each structuredlight pattern. Then the process moves to the next projection unit 112 b.The second projection unit 112 b projects the first structured lightpattern within the scan volume 112 and onto the surface of the handpositioned therein. Each of the one or more cameras 110 captures animage of the hand from its respective field of view. This processcontinues until the second projection unit 112 b has projected eachstructured light pattern needed for the SLI technique and the cameras110 have captured an image of the hand with each structured lightpattern. The process is continued for a third projection unit 112 c orany other projection units 112 that may be implemented in the biometricssystem 100.

The operation of another embodiment of the biometrics system 100 is nowexplained in more detail with respect to FIG. 10 b. In FIG. 10 a,multiple projection units 112 sequentially project a required structuredlight pattern while cameras 110 capture images. This embodiment may haveadvantages with multiple projection units that need some time to switchbetween structured light patterns. Thus, it may be faster to allow arotation between projection units 112 to project images. Afterprojection unit 112 a has projected a first structured light pattern,projection units 112 b or 112 c are projecting the first structuredlight pattern and projection unit 112 a may switch to a secondstructured light pattern.

As explained above, various SLI techniques may be implemented within thebiometrics system 100. The SLI technique must be able to attain theoverall hand, finger and thumb shapes as well as fine detail such asfinger ridges and pores. One SLI technique that meets such requirementsis called multi-frequency Phase Measuring Profilometry (PMP).Multi-frequency PMP is described in, Veera Ganesh Yalla and L. G.Hassebrook, “Very High Resolution 3-D Surface Scanning usingMulti-frequency Phase Measuring Profilometry,” edited by Peter Tchoryk,Jr. and Brian Holz, SPIE Defense and Security, Spaceborne Sensors II,Orlando, Fla., Vol. 5798-09, (Mar. 28, 2005), which is incorporated byreference herein and Jielin Li, L. G. Hassebrook and Chun Guan,“Optimized Two-Frequency Phase-Measuring-Profilometry Light-SensorTemporal-Noise Sensitivity,” JOSA A, 20(1), 106-115, (2003), which isincorporated herein.

FIG. 11 a and 11 b illustrates a novel approach to multi-frequency PMPtechnique 500 that may be used in one embodiment of the biometricssystem 100. Though FIG. 11 only illustrates a single projection unit 112and camera 110, a person of skill in the art would understand thatmultiple projectors and cameras may be used as described above. Thebasic PMP technique projects shifted sine wave patterns onto the 3Dobject, such as thumb 508 and captures a deformed fringe pattern foreach phase shift. The projected light pattern is expressed as:

I _(n)(x ^(P) ,y ^(P))=A ^(P) +B ^(P) cos(2

fy ^(P)−2

n/N)

where A^(P) and B^(P) are constants of the projector, f is the frequencyof the sine wave and (x^(P),y^(P)) is the projector coordinate. Thesubscript n represents the phase-shift index. The total number of phaseshifts is N. FIG. 11 b shows an example of PMP base frequency patterns510 with four phase shifts, e.g. N=4. Thus, as seen in FIG. 11 b, thereare four different phase shifts 512 through 518 of the sine wave at abase frequency. The camera 110 captures an image with each of thepatterns projected onto the 3D object. The captured image is distortedby the 3D object topology and such distortions can be analyzed todetermine the phase value and then the depth changes or worldcoordinates of the 3D object at each pixel of the image can bedetermined. The sine wave pattern is designed so the depth changes arenot affected by perspective by using epipolar lines and a special methodof rectification that minimizes the affects of perspective. So with thephase shifts in the sine wave patterns caused by ridge depth variationas well as average surface depth, the phase of the sine wave pattern isunwrapped into a continuous, nonrepeating phase value across the entiresurface within each partition.

Multi-frequency PMP is derived from the single frequency PMP techniquedescribed above. In multi-frequency PMP, f_(i) different frequencies areprojected, where i=2 to N_(f) and N_(f)=number of frequencies to beprojected. At each of the f_(i) different frequencies, N different phaseshifts are projected. As seen in FIG. 11 a, two different frequencypatterns are projected in one embodiment of the present invention—a basefrequency pattern 504 and high frequency pattern 506. For example, asseen in FIG. 11 a, the base frequency f equals 1, while the highfrequency f equals 4. For each of the frequencies, N phase shifts areused, where N≧2. The low or base frequency sine pattern 504 is optimalfor capture of overall hand shape, finger and thumb shapes. The highfrequency sine pattern 508 is optimal for capture of finer details, suchas finger print ridges and pores. The high frequency sine pattern 508must have a resolution sufficient to extract finger print ridgeinformation. Sine wave patterns also have an advantage that theyeffectively further extend the depth of focus of the system. Digitalcameras use relatively small sensors and thus have lenses with shortfocal lengths and correspondingly large depth of focus. A blurred sinewave pattern remains as a sine wave but with an addition of a DCcomponent. The DC component effectively decreases SNR but the 3D surfacecan still be acquired. Though FIG. 11 a only illustrates two frequenciesf used in the multi-frequency PMP technique, a person of skill in theart would appreciate that other frequencies or only a single frequencymay also be used in other embodiments of a biometric system 100.

In addition, to the two PMP patterns, a third albedo pattern isprojected, wherein the third albedo pattern is a plain white image asseen in FIG. 11 a. The albedo pattern or white pattern is used tocapture texture or albedo of the hand surfaces without a structuredlight pattern. The albedo image serves an important role. The albedovalue is proportional to the reflectivity of the surface, e.g. thefraction of light striking a surface that is reflected by that surface.The albedo image helps determine a percentage of the sine wave patternthat will be reflected and how much will be absorbed by the skin surfaceand hence how bright the structured light pattern will be at aparticular point on the skin surface. As such, the albedo variation canbe removed from the structured light image and from the brightness ofthe sine wave pattern, so that the phase value of the structured lightpattern at a particular point on the skin surface can be determined.

In another embodiment of the present invention, the multi-frequency PMPpatterns in FIG. 11 a have a different color. In this multi-frequency,multi-color PMP pattern technique, each color channel has its ownfrequency, and then each frequency would be shifted by the desirednumber of phase shifts N. For example, the base frequency structuredlight pattern 504 would be red, and N red patterns with N phase shiftsat the base frequency would be projected by the projection unit 112.Then, the high frequency structured light pattern 508 would be green,and N green patterns with N phase shifts at the high frequency would beprojected by the projection unit 112. The multi-color PMP pattern has anadvantage because the spatial frequency f is constant for a given colorchannel, the surface albedo has no affect on the recovered phase value.

There can be many different configurations or combinations in themulti-frequency, multi-color PMP pattern technique. For example, N>3phase shifts with color encoding of three frequencies can be used as acolor encoding techniques but that is relatively independent of surfacecolor. In this case in particular 3 frequencies are encoded into the RGBpixel space where R may contain the base frequency of f=1 and G and Bwould contain the pattern sequence for higher frequencies. Since PMPtechnique is insensitive to color, then within each color space, thereconstructed phase result would also be insensitive to the surfacecolor and albedo.

In another the multi-frequency, multi-color PMP pattern technique, thenumber of phase shifts N=3 with color encoding of 3 frequencies. Thethree color channels are preferably Red, Green and Blue but in theorycan be a large number of spectral components. With 3 color channels thenany 9 pattern combination can be used with color encoding.

In another embodiment, the PMP technique can be applied to a linear rampor triangle waveforms, rather than sine waves. An example of three colortriangle PMP technique 520 is shown in FIG. 11 c. In this example, thereare three patterns or frequencies used each with N=2 phase shifts. Thefirst color 1 is a low frequency triangle or ramp with a positive slopein the first structured light pattern 522 from zero to one intensity anda negative slope ramp in the second structured light pattern 524 withone to zero intensity. The ramp waveform can be modeled as a partialtriangle waveform at a low frequency. In second color 2, the firstpattern 526 is a triangle waveform with a first phase and the secondpattern 528 is the triangle waveform with a second phase. The thirdcolor 3, the first pattern 530 is a triangle waveform with a first phaseand the second pattern 532 is the triangle waveform with a second phase.The frequency f of color 1 is at the low or base frequency. In theexample of FIG. 11 c, the color 1 has a frequency f equals 0.5 of atriangle waveform, the color 2 is at a mid frequency, e.g. f equal 1while color 3 is at a high frequency, e.g. f equals 2.

In this ramp/triangle PMP technique, as few as two gradient ramps, eachwith opposite slopes, can be used to decode a unique phase value andalbedo value. The intensity difference at any point will give the phasevalue and the intensity sum at any point will give the average albedovalue of the surface. With shorter spatial periods the ramps becometriangles waveforms, or repeating ramps. So, one color may be a lowfrequency or single light intensity ramp across the field of view. Thenext color would be sharper ramps that repeat and the third color wouldbe even higher frequency gradient ramp intensity waveforms. Just likePMP, the low frequency is used for non-ambiguous decoding of the phaseand the higher frequency, with sharper gradients are used to moreaccurately detect ridge height variations. The ramp/triangle wave formhas advantages over the sine wave because the sharp edge or tip of thetriangle or ramp provides a good point to establish albedo values.Though only two phase shifts are shown in FIG. 11 c, it may be preferredto have three or more phase shifts at each color channel or frequency.

FIG. 11 d illustrates another embodiment of a color ramp/triangle PMPtechnique 540. In this embodiment, the blue color pattern is a simplerpattern. In experiments and trials conducted, it has been determinedthat the blue color is attenuated by human skin to a much higher degreethan red or green colors. Thus, rather than having a sine wave, ramp,triangle or other pattern, only a very simplified on/off or binarypattern is projected in the blue color channel. As seen in FIG. 11 d,the first blue pattern 534 has a first half color intensity of “0” ordark or “off” and the second half is a light intensity of “1” or “on”.In the second blue pattern 536, the first half has a color intensity of“1” or “on” and the second half is “0” or dark. Since the blue color isattenuated to a high degree in comparison to other colors, it isrecommended to have a simple pattern or binary pattern in comparison tothe other colors green and red.

In another embodiment, weighted combinations of Red, Green and Blue canbe used to encode more than 3 patterns into 3 colors. Theoretically 3colors can be used to contain up to 256̂3 unique colors. However apractical combination may contain 8 distinguishable colors. Thus, in asingle projection pattern, there may be 8 separate component patterns.To improve this further, a second projection pattern, such as albedopattern 502 in FIG. 11 a, could be used to recover the albedo which canbe used to counteract the weighting effects of the surface color andseparate out the reflected component patterns.

In fact, the albedo pattern 502 can be used in addition to any of theabove techniques to recover the albedo values and also for colorcorrection of the color encoded patterns. The albedo pattern can be usedto determine the color ratios of the hand surface. That is if the handsurface has Red, Green, and Blue ratios as 1, 1.5 and 0.2, then thecolors in the structured light patterns can be compensated by scaling by1, 1/1.5 and 1/0.2 respectively.

Though the above description of the SLI technique included multi-colorand multi-frequency PMP patterns along with an albedo pattern, other SLItechniques may be implemented, such as a single structured light patternmay be implemented. A single structured light pattern would increase thespeed of acquisition of the handprint. Such a single structured lightpattern for example may be a composite image comprising a plurality ofmodulated structured light patterns, as described in U.S. patentapplication Ser. No. 10/444,033, entitled, “System and Technique forRetrieving Depth Information about a Surface by Projecting a CompositeImage of Modulated Light Patterns,” by Laurence G. Hassebrook, Daniel L.Lau, and Chun Guan filed on May 21, 2003, which is incorporated byreference here. Alternatively, a simple sine wave pattern in one of theimage color components, such as the “Green” color component may be usedfor the SLI technique while the “Red” and “Blue” color components areused to extract the albedo values.

Alternatively, three colors such as red, green and blue can be used tocreate up to 256³ unique color patterns. However, a practicalcombination may contain 8 distinguishable colors. Thus, in a singleprojection pattern, there may be 8 separate component patterns. Toimprove this further, a second projection pattern could be used torecover the albedo which can be used to counteract the weighting effectsof the surface color and separate out the reflected component patterns.

FIG. 12 illustrates one embodiment of the image processing step 222 fromFIG. 6. Once viable images are captured, the handprint images areprocessed to obtain equivalent 2D rolled ink fingerprint images. In thefirst step 702 of FIG. 12, the handprint images are cropped into desiredpartitions for processing. To ease processing, each complete image orfield of view from each camera may not be processed in its entirety. Thehandprint images may be cropped into partitions that include the areasof the handprints desired for the particular application. For example,FIG. 13 illustrates an image of a handprint 600 with partitions. Thepartitions 602, 604, 606, 608 of the fingerprints and partition 610 ofthe thumbprint may be cropped for processing. These partitions may bestitched together or aligned with partitions from other images as well.The background pattern 310 can be used to quickly identify location ofthe fingertips, thumb or other areas and the desired coordinates of thepartitions.

In step 704 of FIG. 12, calibration parameters and transformationcoefficients are updated based on the known reference measurements ofthe background pattern 310 shown in the handprint images. In step 706,the albedo images are processed to determine average albedo value orcolor intensity of each pixel.

In step 708, the 3D coordinates of hand in the handprint images aredetermined. The 3D world coordinates (x,y,z) of each pixel in thehandprint images with respect to the reference plane in the partitionsof the images is determined. Sample calculations based on dual frequencyPMP sine wave patterns are illustrated in U.S. patent application Ser.No. 10/444,033, entitled, “System and Technique for Retrieving DepthInformation about a Surface by Projecting a Composite Image of ModulatedLight Patterns,” by Laurence G. Hassebrook, Daniel L. Lau, and Chun Guanfiled on May 21, 2003, which is incorporated by reference here. Byprocessing the distortion shown using one or more of the above SLItechniques described above, the phase value at each pixel or point ofthe image, combined with identified world coordinates on the backgroundpattern 310, is transformed to world coordinates of the hand surface.All the fields of view or needed partitions of fields of view aretransformed in this manner into world coordinates sharing the same frameof reference.

In step 710, the overlapping partitions are merged using known worldcoordinates of the background pattern 310. Using the background pattern310, the relative positions of the fingers, thumb and other part of thehand in the images is known with respect to the background pattern 310,and so partitions and images can be aligned based on the backgroundpattern 310. Overlapping fields of view in a partition are analyzed formisalignment and are then corrected resulting in one continuous 3Dsurface representation of the hand or portions of the hand processed.Other methods, such as Iterative Closest Point algorithm may also beemployed to merge the overlapping partitions. Any misalignments arecorrected in step 712. For example, distortions such as barreldistortions or radial distortions may cause misalignment and must becompensated for to correct such misalignments. Once the partitions arestitched together, the 3D model of the handprint surface is completed.The 3D model includes the x,y,z coordinates at each pixel of the surfaceas well as the albedo value or average intensity information at eachpixel.

In step 714, a smooth or average approximation of a handprint surfacefor each partition without ridges or other fine details is determined.By finding surface normal vectors for all the pixels in the average orsmooth approximation of the handprint surface and comparing them withthe 3D world coordinates (x,y,z) of a ridge, detailed handprint surfaceinformation can be extracted. The detailed handprint surface informationincludes the shape and height or depth of the ridges with respect to theaverage approximation of the handprint surfaces. Thus, the handprintridge heights are determined when the detailed handprint surfaceinformation is extracted in each partition.

In step 716, the 3D model is unwrapped into a 2D flat surface. Thesmooth or average approximation of the handprint surface is mapped to a2D rolled print data space. In this process, the average approximationof the handprint surface is warped to a flat 2D surface analogous torolling an inked finger. In one embodiment of the invention to achievethe rolled equivalent, a rectangular mesh of nodal points connected withvirtual springs is generated having a relaxation distance equal to theEuclidean distance between two points in the 3-D space. These nodalpoints in the rectangular mesh are taken from a set of all or less thanall of the smooth or average approximated surface points obtained instep 714. These points are then projected on a 2-D surface and areallowed to iteratively expand thereby reducing the total energy builtinto each spring. The extracted handprint surface from step 714 is thenwarped onto the resulting nodal points, which can then be interpreted asthe rolled equivalent of a handprint. The details of the processing insteps 714 and 716 are explained below with respect to FIG. 14.

In step 718, the ridge height information from the extracted handprintsurface is translated into a grey scale, so that depths and heights offinger print ridges are represented in the image by the grey scale. Theridge height information or the difference vector values are mapped to agray-scale image index value such as 8 or 16-bits per pixel. The 3Drepresentation of the handprint is thus transformed into a 2Drepresentation equivalent of a rolled inked handprint. The 2D rolledequivalent handprint images may be formatted or rendered into differentfile types or standards as specified by an application or industrystandard, in step 720. For example, to meet certain industry standards,the 2D rolled equivalent handprint images must be properly compressed,demographic data included in the correct format. The formatting of thefiles for the 2D rolled equivalent handprint images ensures that theresulting files are constructed properly for interoperability withgovernment agencies or other standard compliant vendors. Commercialdevelopment tools, such as Aware's NISTPACK too can be used to helpgenerate such standard compliant fingerprint data files.

FIGS. 14 a-d provide in more detail the processing steps 714 and 716 ofFIG. 12. The method of creating a 2D rolled equivalent handprint in oneembodiment of the biometrics system 100 is shown in FIG. 14 a. Themethod in FIG. 14 is illustrated with respect to one fingertippartition, but a person of skill in the art would appreciate that themethod may be applied to other partitions showing other parts of thehandprint and/or to other biometric features.

In the first step 802, a smooth fingerprint surface is extracted thatapproximates the fingerprint shape. Specifically, the surface extractionalgorithm virtually peels the surface characteristics off the 3-D scanby smoothing the fingerprint ridges in the 3-D scans. The resultingsmooth surface is a reconstructed manifold that closely approximates thefinger's shape. Various algorithms have been proposed in the literaturefor smoothing and rendering 3-D point clouds but each has disadvantages,such as extended processing. In this embodiment of the biometrics system100, the method 800 uses orthogonal regression planes to do the surfaceapproximation. At each 3-D point in the dataset, a plane is fitted to asubset of points defined by a W×W sized kernel centered at that point. Aweighted nonlinear, least-squares method is used to fit the plane,giving more weight to points near the point whose response is beingestimated and less weight to points further away, i.e. if the localsurface is given by S, then,

${S = {\min\limits_{a}{\sum\limits_{i = 1}^{N}{w_{i} \cdot \left( {f\left( {a,x_{i},{y_{i}z_{i}}} \right)} \right)^{2}}}}},{Where}$f = a(1)x + a(2)y + a(3)z + a(4)

and wi is the weight of the ith residual. To achieve a non-linearfitting, iterative optimization procedures are applied to estimate theplane parameter values till convergence is achieved. The use ofiterative procedures requires starting values to be provided foroptimization. The starting values must be reasonably close to theunknown parameter estimates or the optimization procedure may notconverge. A good approximation of these starting values is calculatedusing the Singular Value Decomposition (SVD) method. The centroid of thedata and the smallest singular value, obtained from the SVD method,defines the initial plane that is used for the optimization process. Theweights assigned to each point for determining its influence on thefitting are calculated using a Gaussian function,

w _(i) =e ^(−disti) ² ^(/σ) ²

where disti is the Euclidean distance between the kernel center pointand ith point of the kernel. The Euclidean distance is calculatedbetween the orthogonal projections of the points onto the fitted plane.The advantage of using this technique is that the points get weightedaccording to their actual position on the fingerprint surface and not onthe basis of their location in the scanner space. This helps extract thefingerprint surface from the 3-D scan with utmost fidelity and accuracy.The orthogonal projection is computed by calculating the point ofintersection of the plane and the perpendicular line through therespective point to the plane. The variance, σ, is a user-definedparameter that controls the degree of approximation or the amount ofsmoothing with the larger variance value leading to the smoothersurface. FIG. 14 b illustrates the original 3D surface scan 820, and theaverage or smooth approximated surface 822 obtained after smoothing withvariance value σ² equal to 0.01. Lower values of the variance σ² willprovide a surface that is less smooth and still has some detectableridges while higher values may provide too much smoothness and lose someof the shape of the finger.

In step 802, after obtaining the smooth or average approximation to thefingerprint shape using the above algorithm, the fingerprint surface isextracted by subtracting the smooth or average approximated surface fromthe original 3D surface scan. The difference vector between the original3-D scan and this smooth or average approximated surface gives therequired fingerprint surface. The detailed fingerprint surfaceinformation is obtained by taking the magnitude of the differencevector, the sign of which depends on the sign of the difference vectorin the Z direction. FIG. 14 b shows an example of an extractedfingerprint surface 824 warped as the color component on the smoothened3-D surface. The extracted handprint surface 824 is essentially thedifference surface between the original 3-D scan and the smoothenedmodel. The height or depth of the handprint ridges can be determinedwith respect to the smooth or average approximated surface. Thus, thedetailed handprint surface information from the extracted handprintincludes the handprint ridge heights in each partition.

In the next step 806, the 3D model is unwrapped into a 2D flat surface.The 3-D fingerprint surface needs to be flattened to get 2-D rolledequivalent fingerprint image. These images are generated by applying asprings algorithm. The springs algorithm establishes a mapping forconverting the initial 3-D shape to a flattened 2-D shape, by applying amass spring system to the 3-D point cloud. The basic idea used is totreat the point cloud as a mechanical system, in which points arereplaced by a body with some mass and these bodies are connected to eachother by springs. All the springs have a relaxed length and a currentlength. If the relaxed length is greater than the current length of thespring, then the spring is compressed between the two bodies and the twobodies need to move apart for the spring to reach its natural length.Similarly if the relaxed length is less than the current length of thespring, the spring is stretched and the two connecting bodies need tocome closer for the spring to attain the relaxed length. The springforces will attract or repulse the bodies until the system reachesminimum energy. This is called the balanced state.

For flattening, first a rectangular mesh of nodal points connected bysprings is generated from a subset of the point cloud or 3D coordinatesof the smooth or average approximated surface, as shown in FIG. 14 astep 806. The virtual springs have a relaxation distance equal to theEuclidean distance between two points of the 3D space of the handprintimages. In step 808, these points are then projected onto a 2D surfacein step 810. In step 812, the points are allowed to iteratively expandthereby reducing the total energy built into each string. The energystored in the virtual springs connecting a point in the mesh to the8-connected neighbors has to be minimized. To minimize this energy, onlythe point whose displacement is being calculated moves and the remainingpoints remain fixed. The displacement of the point is iterative andevery iteration consists of one pass over all the points in the mesh. Toevaluate the total energy e at a point, the energy stored in each springconnecting the point to its neighbors is added together,

$e = {\sum\limits_{i = 1}^{n}e_{i}}$

The individual energy e_(i) is computed by squaring the magnitude of thedisplacement between the current length of the spring and its relaxedlength. The sign of the displacement vector determines the type offorce, attractive or repulsive that has to be applied to the spring inorder to achieve the balanced state. The energy stored in the ith springis hence determined by,

e _(i)=sign(d _(i) −r _(i))·(d _(i) −r _(i))²

where, d_(i) is the current length which is taken to be the Euclideandistance between the points in the 2-D space and r_(i) is the relaxedlength of the i^(th) spring determined by the Euclidean distance betweenthe points in the 3-D space. The energy in each spring is then assignedthe direction of the displacement vector and added to the total energy eat the point under consideration. To attain the equilibrium state, thepoint has to move depending on the energy stored at that point. Apercentage amount, λ, of the total energy is used to displace the meshpoint in the 2-D space. The value of λ must be chosen to prevent makingthe system unstable, e.g. large values of λ can make the systemunstable. FIG. 14 c shows a simulation 830 of the algorithm inone-dimensional space in the X-Y plane. The points marked as stars (*)are the initial 2-D nodal mesh given as input to the springs algorithm.The points marked as blank circles (∘), are positions of the same pointsafter 1000 iterations of the Springs algorithm. As the number ofiterations increase, the points move in the 2-D space to attain anequilibrium state. The balanced state is achieved when the distancebetween these points is equal to their Euclidean distance in the 3-Dspace.

FIG. 14 d illustrates the 2D rolled equivalent fingerprint 840 generatedfrom the unraveled fingerprint surface, obtained by applying the springsalgorithm to the fingerprint scan shown in FIG. 14 b. The 2-D unravelednodal points in FIG. 14 d were obtained after 5,000 iterations. Sinceonly a subset of the 3D coordinates of the original handprint scan wereassigned as nodal points in the mesh, any of the other unassigned 3Dcoordinates may be assigned within the 2D mesh of unraveled nodalpoints.

In step 814, the extracted fingerprint surface, obtained from theextraction step 804, is warped as a color component onto the 2-D nodalmesh to generate the 2D rolled equivalent fingerprint in FIG. 14 d. Eachof the points in the 2D mesh of unraveled nodal points are assigned aridge height information from the fingerprint surface extracted in step802. Any other detailed fingerprint surface information extracted instep 802 can be warped around as the color component as well. To obtainpoints or pixels at regular intervals, the resulting 2D unraveled meshof nodal points with detailed handprint surface information may besampled along a grid lattice to create the 2-D rolled equivalent image.

Histogram manipulation may also be applied to match the fingerprinthistograms of existing, high quality scans within a target database.This histogram manipulation may be done during the processing to aid infurther processing or at the end of the processing to match thefingerprint with an identity.

The embodiments of the biometrics system 100 described herein havevarious advantages over the prior art. For example, speed of acquisitionof the handprint images is greatly increased from the 5-10 minutes ofrolled ink fingerprinting. Translucent sweat or oil will not corrupt thehandprint images nor will common variations in skin color. Thebiometrics system 100 is robust to translucent materials and resistantto specularity of shiny surfaces. The biometrics system 100 is morerobust to extremely worn ridges of the fingers and palm.

Though the present embodiment has been described for obtaining a handprint, a person of skill in the art would appreciate that the system maybe modified for obtaining images for other biometrics, such as scars,tattoos, facial features, etc. The present system may also be used inother fields besides biometrics. For example, in medical fields, thepresent invention may be used for measuring moles on skin surfaces orother features that need to be recorded, measured or monitored. Otheruses include Human Computer Interaction, prosthetic development,industrial inspection, special effects and others.

While certain representative embodiments have been described herein, aperson of skill in the art would appreciate that various substitutions,modifications or configurations other than those described herein may beused and are within the scope of the claims of the present invention.

What is claimed is:
 1. A method for a processing device to determine atwo dimensional (2D) handprint image from a three dimensional (3D) dataof the handprint, comprising: processing one or more handprint imagescaptured with a structured light illumination technique to determine 3Dcoordinates in the one or more handprint images; extracting by theprocessing device handprint surface information using the 3Dcoordinates, wherein the handprint surface information includes ridgeheight information; generating a 2D handprint image from a set of the 3Dcoordinates; mapping by the processing device the handprint surfaceinformation onto the 2D handprint image; and translating the ridgeheight information to a grey-scale image index.
 2. The method of claim1, wherein extracting by the processing device handprint surfaceinformation using the 3D coordinates of the handprint comprises:generating a smooth handprint surface that approximates a shape of thehandprint; determining surface normal vectors at a plurality of pointsin the smooth handprint surface; and comparing the surface normalvectors at the plurality of points in the smooth handprint surface withthe 3D coordinates in the one or more handprint images to determine thehandprint surface information at the plurality of points.
 3. The methodof claim 2, further comprising: calculating a magnitude of a differencevector between the 3D coordinates of one of the plurality of points inthe one or more handprint images and a corresponding point in the smoothhandprint surface to determine the ridge height information.
 4. Themethod of claim 1, wherein generating the 2D handprint image from theset of the 3D coordinates, comprises: generating by the processingdevice a mesh of nodal points from the set of the 3D coordinates,wherein the mesh of nodal points are modeled as connected by springs;projecting by the processing device the mesh of nodal points onto a 2Dplane; and iteratively allowing the nodal points to move in the 2D planesuch that the total energy stored in the springs connecting the nodalpoints is minimized to generate the 2D handprint image.
 5. A processingdevice for generating a two dimensional (2D) print image fromthree-dimensional (3D) data, comprising: at least one processor operableto: determine 3D coordinates of a print; generate 3D coordinates of asmooth print surface that approximates a shape of the print; extractprint surface information by comparing a set of the 3D coordinates ofthe smooth print surface with the 3D coordinates of the print, whereinthe print surface information includes ridge height information;generate a 2D print image from the set of the 3D coordinates of thesmooth print surface; map the print surface information onto the 2Dprint image; and translate the ridge height information to a grey-scalein the 2D print image.
 6. The processing device of claim 5, wherein theprocessor is operable to generate the 2D print image from the set of 3Dcoordinates of the smooth print surface by: generating a mesh of nodalpoints from the set of the 3D coordinates of the smooth print surface;projecting the mesh of nodal points with 3D coordinates onto a 2D plane;and assigning the print surface information to nodal points in the meshof nodal points projected onto the 2D plane.
 7. The processing device ofclaim 5, wherein the mesh of nodal points are modeled as connected bysprings with a relaxation variable between the nodal points.
 8. Theprocessing device of claim 7, wherein the processor is further operableto: iteratively move the nodal points in the mesh of nodal onto the 2Dplane to lower the relaxation variable in the springs connecting thenodal points to project the 3D coordinates of the mesh of nodal pointsonto the 2D plane.
 9. The processing device of claim 8, wherein theprocessor is further operable to: map 3D coordinates of the smoothhandprint surface not assigned to the mesh of nodal points to the 2Dplane.
 10. The processing device of claim 8, wherein the processor isfurther operable to: resample the nodal points in the mesh of nodalpoints projected onto the 2D plane with print surface information alonga grid lattice.
 11. The processing device of claim 5, wherein the set ofthe 3D coordinates of the smooth print surface is less than all the 3Dcoordinates of the smooth print surface.
 12. The processing device ofclaim 5, wherein the processor is operable to extract print surfaceinformation by comparing a set of the 3D coordinates of the smooth printsurface with the 3D coordinates of the print by: calculating a magnitudeof a difference vector between the 3D coordinates of a point in theprint with a corresponding point in the smooth print surface todetermine ridge height information.
 13. A method for a processing deviceto generate a two dimensional (2D) print image from three-dimensional(3D) data, comprising: determining 3D coordinates of a print; generating3D coordinates of a smooth print surface that approximates a shape ofthe print; extracting print surface information by comparing a set ofthe 3D coordinates of the smooth print surface with the 3D coordinatesof the print, wherein the print surface information includes ridgeheight information; generating a 2D print image from the set of the 3Dcoordinates of the smooth print surface; mapping the print surfaceinformation onto the 2D print image; and translating the ridge heightinformation to a grey-scale in the 3D print image.
 14. The method ofclaim 13, wherein generating the 2D print image from the set of 3Dcoordinates of the smooth print surface includes: generating a mesh ofnodal points from the set of the 3D coordinates of the smooth printsurface; projecting the mesh of nodal points with 3D coordinates onto a2D plane; and assigning the print surface information to nodal points inthe mesh of nodal points projected onto the 2D plane.
 15. The method ofclaim 14, wherein the set of the 3D coordinates of the smooth printsurface is less than all the 3D coordinates of the smooth print surface.16. The processing device of claim 13, wherein extracting print surfaceinformation by comparing a set of the 3D coordinates of the smooth printsurface with the 3D coordinates of the print includes: calculating amagnitude of a difference vector between the 3D coordinates of a pointin the print with a corresponding point in the smooth print surface todetermine ridge height information.
 17. A processing device operable todetermine a two dimensional (2D) handprint image from a threedimensional (3D) data of the handprint, comprising: at least oneprocessor operable to: process one or more handprint images capturedwith a structured light illumination technique to determine 3Dcoordinates in the one or more handprint images; extract ridge heightinformation using the 3D coordinates; generate a 2D handprint image froma set of the 3D coordinates, wherein the 2D handprint image includes amapping of the ridge height information onto the 2D handprint imageusing a grey-scale index.
 18. The processing device of claim 17, whereinthe at least one processor is operable to extract handprint surfaceinformation using the 3D coordinates of the handprint by: generating asmooth handprint surface that approximates a shape of the handprint;determining surface normal vectors at a plurality of points in thesmooth handprint surface; and comparing the surface normal vectors atthe plurality of points in the smooth handprint surface with the 3Dcoordinates in the one or more handprint images to determine thehandprint surface information at the plurality of points.
 19. Theprocessing device of claim 18, wherein the at least one processor isfurther operable to: calculate a magnitude of a difference vectorbetween the 3D coordinates of one of the plurality of points in the oneor more handprint images and a corresponding point in the smoothhandprint surface to determine the ridge height information.
 20. Theprocessing device of claim 17, wherein the at least one processor isoperable to generate the 2D handprint image from the set of the 3Dcoordinates by: generating a mesh of nodal points from the set of the 3Dcoordinates; projecting the mesh of nodal points onto a 2D plane.